IN THE CLAIMS 
Please amend the claims as follows: 

1. (original) A system (100) for integrative analysis of 
intrinsic (10) and extrinsic (11) audio-visual data, the system 
comprising : 

an intrinsic content analyser, the intrinsic content analyser 
being communicatively connected to an audio-visual source, the 
intrinsic content analyser being adapted to search the audio-visual 
source for intrinsic data and being adapted to extract intrinsic 
data using an extraction algorithm, 

an extrinsic content analyser, the extrinsic content analyser 
being communicatively connected to an extrinsic information source, 
the extrinsic content analyser being adapted to search the 
extrinsic information source and being adapted to retrieve 
extrinsic data using a retrieval algorithm, 

wherein the intrinsic data and the extrinsic data are 
correlated, thereby providing a multisource data structure. 

2. (original) A system according to claim 1, wherein the 
retrieval of the extrinsic data is based on the extracted intrinsic 
data. 



N:\userPublic\NL040163 prelim.doc 



3. (original) A system according to claim 1, wherein the 
extraction and/or retrieval algorithm (s) is/are provided by a 
module . 

4. (original) A system according to claim 1, wherein a query is 
provided by a user, the query being provided to the extraction 
algorithm and wherein the intrinsic data is extracted in accordance 
with the query. 

5. (original) A system according to claim 1, wherein a query is 
provided by a user, the query being provided to the retrieval 
algorithm and wherein the extrinsic data is retrieved in accordance 
with the query. 

6. (original) A system according to claim 1, wherein a feature 
reflected in the intrinsic and extrinsic data include textual, 
audio and/or visual features. 

7. (original) A system according to claim 1, wherein the audio- 
visual source is a film (101) and wherein the extracted data 
include textual (104), audio and/or visual features (105, 106). 



8. (original) A system according to claim 1, wherein the 
extrinsic information source is connected to and may be accessed 
via the Internet (103) . 

9. (original) A system according to claim 1, wherein the 
extrinsic information source is a film screenplay (102) . 

10. (original) A system according to claim 9, wherein the 
extrinsic content analyser include knowledge about screenplay 
grammar, and wherein the extrinsic data is retrieved based on 
information extracted from the screenplay by use of the screenplay 
grammar . 

11. (currently amended) A system according to any of the claims 9 
or lO claim 9 wherein the identification (5) of persons in a film is 
obtained by means of the screenplay. 

12. (currently amended) A system according to any of the claimo 9 
o r lO claim 9 wherein a feature in a film is analysed based on 
information included in the screenplay. 

13. (original) A system according to claim 1, wherein the 
correlation of the intrinsic and extrinsic data is time correlation 
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(121) , thereby providing a multisource data structure where a 
feature reflected in the intrinsic data is time correlated to a 
feature reflected in the extrinsic data. 

14. (original) A system according to claim 13, wherein the time 
correlation is obtained by an alignment of a dialogue (120) in the 
screenplay to the spoken text (104) in the film and thereby 
providing a timestamped transcript (121) of the film. 

15. (original) A system according to claim 14, wherein a speaker 
identification in the film is obtained from the timestamped 
transcript . 

16. (original) A system according to claim 9, wherein the 
screenplay is compared with the spoken text in the film by means of 
a self -similarity matrix (30) . 

17. (original) A system according to claim 1, wherein a high- 
level information structure (5-9) is generated in accordance with 
the multisource data structure. 

18. (original) A system according to claim 17, wherein the high- 
level information structure may be stored on a storage medium. 
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19. (original) A system according to claim 17, wherein an updated 
high-level information structure is generated, the updated high- 
level information structure being an already existing high-level 
information structure which is updated in accordance with the 
multisource data structure. 

20. (original) A system according to claim 1, wherein the 
retrieval algorithm is a dynamic retrieval algorithm adapted to 
dynamically update itself by including additional functionalities 
in accordance with retrieved extrinsic data. 

21. (original) A system according to claim 20, wherein the 
additional functionalities is obtained by training the retrieval 
algorithm on a set of features from intrinsic data using labels 
obtained from the extrinsic data. 

22. (currently amended) A system according to claim 9 and 21 , 
wherein the training is performed using at least one screenplay. 

23. (original) A system according to 1, wherein an automatic 
ground truth identification in a film is obtained based on the 



N:\userPublic\NL040163 prelim.doc 



multisource data structure for use in benchmarking algorithms on 
audio-visual content. 



24. (original) A system according to 1, wherein an automatic 
scene content understanding in a film is obtained based on the 
textual description in the screenplay and the audio-visual features 
from the film content. 

25. (original) A system according to 1, wherein an automatic 
labelling in a film is obtained based on the multisource data 
structure . 

26. (original) A method for integrative analysis of intrinsic and 
extrinsic audio-visual information, the method comprising the steps 
of: 

searching an audio-visual source for intrinsic data and 
extracting intrinsic data using an extraction algorithm, 

searching an extrinsic information source and retrieving 
extrinsic data using a retrieval algorithm, 

correlating the intrinsic data and extrinsic data, thereby 
providing a multisource data structure. 



27. (original) A method according to claim 26 further comprising 
the step of generating a high-level information structure in 
accordance with the multisource data structure. 

28. (original) A method according to claim 26, wherein the 
extrinsic content analyser include knowledge about screenplay 
grammar, and wherein the extrinsic data is retrieved using 
information extracted from the screenplay by use of the screenplay 
grammar . 

29. (original) A method according to claim 26, wherein the 
retrieval algorithm is updated by training the algorithm on a set 
of extrinsic data. 

30. (canceled) 

31. (canceled) 

32. (canceled) 
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